The user as wizard: A method for early involvement in the design and evaluation of adaptive systems

User testing tends to put participants in the role of the user. To evaluate complex systems (such as adaptive systems), a Wizard-of-Oz study may be used, where the role of the system is made easier by allowing somebody on the design team to perform part of it. In this paper, we propose a method whereby participants take the role of the system, allowing early user involvement in the design process. Participants’ actions can inspire the design of adaptive algorithms, and their rationale can inspire evaluation criteria. We illustrate the method with a number of case studies.